1 Student Growth Percentile Fit Plots

A goodness of fit plot is produced for each unique analysis run in 2015. Each analysis is defined by the unique grade and content area combinations. Most fit plots contain four panels. When the prior scale score is unavailable the top panel will be excluded. Usually unavailability is due to the use of equated SGP estimation in an assessment program transition year or from an End of Course Test (EOCT) analysis that uses a prior course progression that is not a subset of the most typical (i.e. “canonical”) course progression. The top panel is a mosaic plot that shows the percentage of students that fall into each prior proficiency level, and the location of the 10^th through 90^th quantiles of the Student Growth Percentile (SGP) distribution represented as dashed white lines (with the exception of the solid white line for the median/50^th percentile). Ideally the median SGP will be at or near 50 for all prior achievement level groups.

The “Ceiling/Floor Effects Test” panel helps identify problems in SGP estimation at the Highest and Lowest Obtainable (or Observed) Scale Scores (HOSS and LOSS). If is is relatively typical for extremely high (low) achieving students to consistently score at or near the HOSS (LOSS) each year, the SGPs for these students may be unexpectedly low (high). That is, if a sufficient number of students maintain performance at the HOSS over time, this performance will be estimated to be typical, and therefore SGP estimates will reflect typical growth (e.g. 50th percentile). In some cases small deviations from these extreme score values might even yield low growth estimates. Although these score patterns can legitimately be estimated as typical or low growth percentiles, it is potentially an unfair description of student growth (and by proxy teacher or school, etc. performance). Ultimately this is an artifact of the assessments’ inability to adequately measure student performance at extreme ability levels.

The table of values shows whether the current year scale scores at both extremes yield expected SGP values. The expectation is that the majority of SGPs for students scoring at or near the LOSS will be low (preferably less than 5 and not higher than 10), and that SGPs for students scoring at or near the HOSS will be high (preferably higher than 95 and not less than 90)¹. Appendix C provides a more detailed analysis of potential ceiling and floor effects.

The bottom left panel shows the empirical distribution of SGPs given prior scale score deciles in the form of a 10 by 10 cell grid. Percentages of student growth percentiles between the 10^th, 20^th, 30^th, 40^th, 50^th, 60^th, 70^th, 80^th, and 90^th percentiles were calculated based upon the empirical decile of the cohort’s prior year scaled score distribution². Deviations from perfect fit are indicated by red and blue shading. The further above 10 the darker the red, and the further below 10 the darker the blue. A more detailed discussion about the reasons for and implications of model misfit for the various SGP analysis types can be found in the “Goodness of Fit” section of the 2015 New Jersey Student Growth Model report.

The bottom right panel of each plot is a Q-Q plot which compares the observed distribution of SGPs with the theoretical (uniform) distribution. An ideal plot here will show black step function lines that do not deviate greatly from the ideal, red line which traces the 45 degree angle of perfect fit.

2 SAGE Grade-Level Fit Plots

3 SAGE EOCT Fit Plots